The risk analysis index is an independent predictor of outcomes after lung cancer resection

Background The Risk Analysis Index (RAI) is a frailty assessment tool based on an accumulation of deficits model. We mapped RAI to data from the Society of Thoracic Surgeons (STS) Database to determine whether RAI correlates with postoperative outcomes following lung cancer resection. Methodology/Principal findings This was a national database retrospective observational study based on data from the STS Database. Study patients underwent surgery 2018 to 2020. RAI was divided into four increasing risk categories. The associations between RAI and each of postoperative complications and administrative outcomes were examined using logistic regression models. We also compared the performance of RAI to established risk indices (American Society of Anesthesiology (ASA) and Charlson Comorbidity Index (CCI)) using areas under the Receiver Operating Characteristic (ROC) curves (AUC). Results: Of 29,420 candidate patients identified in the STS Database, RAI could be calculated for 22,848 (78%). Almost all outcome categories exhibited a progressive increase in marginal probability as RAI increased. On multivariable analyses, RAI was significantly associated with an incremental pattern with almost all outcomes. ROC analyses for RAI demonstrated “good” AUC values for mortality (0.785; 0.748) and discharge location (0.791), but only “fair” values for all other outcome categories (0.618 to 0.690). RAI performed similarly to ASA and CCI in terms of AUC score categories. Conclusions/Significance RAI is associated with clinical and administrative outcomes following lung cancer resection. However, its overall accuracy as a surgical risk predictor is only moderate and similar to ASA and CCI. We do not recommend routine use of RAI for assessment of individual patient risk for major lung resection.


Introduction
In 2022, there were 236,749 estimated number of new lung cancer cases and 130,180 estimated deaths from lung cancer [1].As the projected number of people 65 years and older grows from 59 million in 2022 to 73 million in 2030 [2], thoracic surgeons are increasingly presented with the task of assessing operative candidacy and performing risk stratification for older patients with lung cancer.Chronological age does not always reflect biological age, and elderly people have a range of functional status that varies from robust to frail.Frailty has been identified as the progressive loss of physical and mental function that leads to an abnormal response to physiologic stressors [3].In surgical fields, frailty correlates well with both mortality and morbidity across a broad spectrum of operations [4][5][6].We recently characterized the 5-factor modified frailty index as predictive of most outcomes after major lung resection, but this metric was only marginally better than more commonly used risk assessment tools such as the American Society of Anesthesiology (ASA) Physical Status Classification System and Charlson Comorbidity Index (CCI) [7].
The original Risk Analysis Index was developed as a measure of frailty and used to predict 6-month mortality in nursing home residents [8].This was later revised for surgical patients (Revised Risk Analysis Index, hereafter referred to as the Risk Analysis Index and abbreviated as RAI), and was externally validated using various national databases [9,10].Despite growing literature on RAI as a predictor of postoperative outcomes in high-risk operations including lung cancer resections [10,11], there has not been any dedicated, in-depth analysis of RAI as a predictor of postoperative outcomes following major lung resection for cancer using the Society of Thoracic Surgery (STS) database.In the present study, we mapped the RAI variables to patient data from the STS General Thoracic Surgery Database and analyzed whether the RAI correlates with postoperative outcomes in patients following lung cancer resection.

Data
The use of a limited dataset from the STS General Thoracic Surgery Database was approved by The University of Chicago Institutional Review Board (IRB21-0399, approved 3/9/2021) and the need for informed consent and consent documentation was waived.The STS General Thoracic Surgery Database contains more than 700,000 general thoracic surgery procedure records and currently has more than 1,000 participating surgeons [12].We queried the STS database (Version 2.41) for all patients with a diagnosis of lung cancer undergoing elective anatomic lung resection from January 1, 2018 through December 31, 2020.Data were collected by the STS database during the period January 1, 2018 through June 30, 2021, and data were accessed July 26, 2021.Data were collected for demographic, physiologic, operative, and outcome variables.Comorbidities and ASA status were also collected.Patients were excluded if they underwent sleeve (carinal) pneumonectomy, extrapleural pneumonectomy, resection of an apical lung tumor including chest wall resection, completion pneumonectomy, chest wall reconstruction with muscle flap, or lung volume reduction surgery, or if they had a history of prior cardiothoracic surgery.Given that the patient data was de-identified, the authors had no access to identifying information during or after data collection.Because of purchase from a third party, the data cannot be made publicly available.Access to the data may be obtained through an application process (S1 Text).

Outcome measures
Complications were categorized as pulmonary, cardiovascular, infectious, neurological, gastrointestinal, urinary, surgical, and in-hospital mortality (S1 Table ).Composite outcomes included major postoperative complications and any postoperative event (S2 Table ).Perioperative administrative outcomes included 30-day mortality, unexpected admission to the intensive care unit, readmission within 30 days of discharge, and discharge location other than home.

Metrics
RAI was assessed by mapping ten variables in the STS database to eleven factors: age, sex, weight loss, poor appetite, congestive heart failure, dyspnea, renal failure, presence of cancer, functional status, cognitive decline, and living status (Table 1).An RAI score from 0 to 81 was calculated for each patient, with an increasing RAI score related to an increasing incidence of frailty.RAI was categorized into four subgroups (�34, 35-39, 40-44, �45) as previously described [13].Patients who did not have all the variables necessary for RAI calculation were included in a "missing" RAI category.The modified CCI was calculated as previously described [14].ASA status was abstracted from STS data, which used a standard definition [15].

Statistical techniques
Preoperative patient characteristics were compared among different RAI categories using Chisquare tests for categorical variables and ANOVA for continuous variables.Multivariable logistic regression models were used to evaluate the association between RAI categories and each of postoperative outcome controlling for extent of resection, induction therapy, body mass index (BMI) category, coronary artery disease (CAD), cerebrovascular disease, forced expiratory volume in the first second expressed as a percent of predicted (FEV1%), diffusing capacity of the lung for carbon monoxide expressed as a percent of predicted (DLCO%), hypertension, diabetes, and pathological T, N, and M stages [16] (S3 Table ).Similar modeling was performed for CCI and ASA.
Predicted probability of each outcome for RAI risk categories was estimated from the multivariable logistic regression model.A progressive increase in the marginal probability as risk scores within a metric increased was interpreted as a desirable characteristic.Monotonicity was classified as strictly monotonic (progression between all ORs), partially monotonic (lack of progression between any two consecutive ORs), or non-monotonic.Receiver operating characteristic (ROC) analysis was performed for each instrument.The area under the ROC curve (AUC) for each pair of the instruments was compared using the DeLong test.All statistical analyses were performed using R software version 3.3.0.

Patients
Of 29,420 patients identified in the STS General Thoracic Surgery Database according to the inclusion criteria and exclusion criteria, 22,848 (77.6%) had all the factors necessary for calculation of RAI (S1 Fig) .Patients who lacked the necessary data to calculate RAI were most similar demographically and clinically to patients with an RAI score of 35-39 or 40-44, particularly with regards to age, gender, living status, shortness of breath, congestive heart failure, dialysis status, functional status, and neurocognitive dysfunction.These preoperative variables were included in the calculation of RAI and correlated positively with RAI (S4 Table ).Among preoperative variables not included in the calculation of RAI, many comorbidities also had clinically significant correlations with RAI.There was no clinically significant change in distribution of BMI across RAI categories.Similarly, no consistent trends were seen with induction therapy with respect to RAI scores.However, patients with higher RAI scores were more likely to have higher pathological and clinical T stages for lung cancer.We compared the cohort that had the data needed for calculation of RAI to the cohort that did not and found no evidence of important clinical differences (S5 Table ).
The median score for ASA and CCI was 3, whereas it was 36 for RAI (Table 2).The RAI and CCI scores were reasonably well distributed among three and four score categories, respectively, with sparse population of the one remaining category for RAI and among the three remaining relevant categories for CCI.Of note, CCI scores were always 2 or greater and RAI scores were always 28 or greater because all patients had a diagnosis of lung cancer.ASA was well distributed among three categories, with the remaining three categories being sparsely populated.

Multivariate analyses
On multivariable analyses, RAI was significantly associated with an incremental pattern with almost all outcomes (Table 3).Multivariate analyses for ASA demonstrated a significant association with almost all postoperative and administrative outcomes with the exception of infectious complications, unexpected ICU admission, and unanticipated surgical approach conversion (S6 Table ).CCI was significantly associated with almost all postoperative and administrative outcomes with the exception of infectious complications (S7 Table ).

Assessment of performance as risk metrics
Monotonicity of marginal probability was strict or partial among all outcome categories for RAI and ASA.Monotonicity of marginal probability was strict or partial for CCI among all outcome categories with the exception of gastrointestinal outcomes.The incidence of strict monotonicity was similar among CCI (11/15), RAI (9/15), and ASA (10/15).Of note, ASA was at less risk for partial monotonicity based on having only 3 categories compared to 4 for RAI and CCI (S8 Table ).ROC analyses demonstrated "good" AUC values (.70 to .80) for all three metrics related to mortality and discharge to other than home, whereas AUC values for other outcomes were in the fair category for all metrics (0.60 to 0.70; Table 4).Comparison of AUC values between each pair of the instruments using the DeLong test showed that RAI outperformed CCI and ASA in 6 of the 15 outcome categories, although the differences were not clinically important.There was no significant difference between CCI and ASA on ROC analyses (Table 4).

Discussion
In many surgical fields, frailty has been shown to correlate with postoperative complications, discharge to institutional care, and mortality [17,18].The original Risk Analysis Index is a tool based on the accumulation of deficits model of frailty derived from the Minimum Set Mortality Risk Index-Revised (MMRI-R) instrument used to predict 6-month mortality in nursing home residents [8].Arya et al. revised the original Risk Analysis Index scoring system for nonveteran surgical patients, externally validated the revised RAI using the National Surgical Quality Improvement Program Database (NSQIP) in patients undergoing elective noncardiac operations, and reported that RAI had improved discrimination and calibration as a frailtyscreening tool in surgical patients compared to the original RAI [9].In contrast, Wan et al. further investigated the accuracy of RAI in predicting postoperative morbidity and mortality in patients undergoing high-risk operations using NSQIP data and reported it to be an ineffective predictor of 30-day morbidity and mortality for patients undergoing high-risk operations including lung resection [11].To date, no study has investigated the predictive value of the revised RAI (hereafter abbreviated as RAI) focusing on patients undergoing lung resection using the using the Society of Thoracic Surgery (STS) database.The current study mapped RAI variables to patient data from the STS database and assessed whether frailty as assessed by RAI is independently associated with postoperative outcomes in patients following anatomic lung resection for lung cancer.We found RAI to be an independent predictor of all postoperative outcomes following anatomic lung section for lung cancer.On multivariable analysis, increasing RAI scores were associated with incremental changes in the odds of adverse outcomes that were clinically meaningful.We also demonstrated that RAI was associated with a wide variety of administrative outcomes.RAI performed similarly to ASA and CCI based on overall accuracy.Contrary to ROC analyses using the NSQIP database which identified RAI as a poor predictor of operative mortality [11], our analyses using the STS database suggested RAI to be a good predictor of postoperative mortality.However, RAI was similarly weak in predicting other postoperative outcomes compared to ASA and CCI, suggesting that its routine use in assessing surgical risk is only of moderate benefit.An additional strength of RAI may be its potential ability to assess frailty.Tools assessing the "frailty phenotype" are sometimes impractical for routine screening in busy surgical clinics because they require time and effort from clinical staff to measure walking speed and grip strength [19].RAI requires no special equipment and can be calculated in less than 60 seconds, allowing easy implementation and resulting in high participation compliance among surgical clinics [20].However, although associations of RAI with surgical outcomes have been demonstrated, and its performance is similar to that of other frailty metrics generated using data from the electronic medical record, correlations between RAI and components of physical frailty such as weight loss, low gait speed, weakness, sarcopenia, and low energy have not been adequately assessed.In our study RAI did not correlate strongly with underweight status, suggesting a weak relationship with weight loss and weakness.Thus, whether RAI merely represents a potentially useful surgical risk tool or in addition has the added benefit of assessing frailty in a lung resection population has yet to be determined.
Although RAI may have the advantage of ease of calculation using the electronic medical record, its design as an accumulation of deficits model precludes extensive use of physical performance metrics for such calculation.Regardless of whether RAI is an accurate measure of frailty, the information from RAI cannot be used to identify individual elements in patients that might be targeted for mitigation to reduce operative risk, and RAI values for an individual patient will be unlikely to decrease in response to such mitigation.Therefore, RAI may serve primarily as a means for improved patient selection for surgery.There are potential limitations to this study.Selecting covariates for multivariate analyses is sometimes challenging in studies such as this.There are a number of known factors that are associated with adverse clinical outcomes after resection for lung cancer that have been derived through assessment of the STS General Thoracic Surgery database [16].Unfortunately, Version 2.41 of the STS data selected for the current study did not capture steroid use and Zubrod performance status, which have been shown to be positively correlated with postoperative mortality.Furthermore, gender and renal dysfunction could not be included in the multivariable model because they were included in the calculation of RAI.Review of the distribution of gender and renal dysfunction across the different RAI categories revealed strong collinearity between RAI and these variables (S4 Table ).To directly compare the performance of ASA, CCI and RAI as surgical risk metrics, we removed ASA from the multivariable model when analyzing RAI and CCI.
There were a large number of missing values related to some outcome and covariate variables within the STS database.The general thoracic surgery STS database currently only receives data from 271 participant sites across North America, with center-level penetration between 40% and 50% [21], thus our findings may not be generalizable to all centers in North America.Data on post-discharge status are limited, largely restricting our analyses to in-hospital outcomes with no survival data.It is likely that a minimally invasive approach may lead to lower complication rates than an open approach, as is being studied in the ongoing VIOLET trial [22].However, we did not assess surgical approach (open, VATS, robotic) and its possible effect on the utility of RAI in predicting complications, as this is the topic of a separate ongoing study using STS database data.Finally, we limited our study to Version 2.41 of the STS database due to changes in collected variables that permitted calculation of the RAI score using only this version.
We conclude that the RAI is associated with clinical and administrative outcomes following lung cancer resection.However, its overall accuracy as a surgical risk predictor is only moderate and similar to ASA and CCI.Our findings invite the question of whether RAI is a good frailty metric or is more accurately a surgical risk predictor in disguise.Future studies will be needed to investigate the relationship between RAI, physical frailty metrics, and sarcopenia.

Table 1 . Society of thoracic surgery database variables mapped to the Risk Analysis Index (RAI). Risk Analysis Index variable Matched variable(s) in STS Database Version 2.4 Risk Analysis Index score Male sex Gender
* All patients included in the study had diagnosis of lung cancer; unintentional weight loss was used twice in matching STS variables to RAI variables because no other STS variable was suitable for matching.https://doi.org/10.1371/journal.pone.0303281.t001